Skip to content

[rllib] Support batch norm layers#3369

Merged
ericl merged 10 commits intoray-project:masterfrom
ericl:batch-norm
Nov 29, 2018
Merged

[rllib] Support batch norm layers#3369
ericl merged 10 commits intoray-project:masterfrom
ericl:batch-norm

Conversation

@ericl
Copy link
Contributor

@ericl ericl commented Nov 21, 2018 •
edited
Loading

What do these changes do?

  • Pass in the right is_training tensor to build_layers_v2
  • Add the right update ops to apply grads automatically
  • Example script in test_batch_norm.py

I don't think this will work for e.g., A3C which applies gradient updates separately, but it should work fine in the other execution modes.

Related issue number

Closes: #2023

@AmplabJenkins
Copy link

AmplabJenkins commented Nov 21, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9498/
Test FAILed.

@ericl ericl added the tests-ok The tagger certifies test failures are unrelated and assumes personal liability. label Nov 21, 2018
@AmplabJenkins
Copy link

AmplabJenkins commented Nov 22, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9528/
Test FAILed.

@ericl ericl removed the tests-ok The tagger certifies test failures are unrelated and assumes personal liability. label Nov 22, 2018
@AmplabJenkins
Copy link

AmplabJenkins commented Nov 22, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9542/
Test FAILed.

@ericl ericl added the tests-ok The tagger certifies test failures are unrelated and assumes personal liability. label Nov 22, 2018
@AmplabJenkins
Copy link

AmplabJenkins commented Nov 24, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9574/
Test FAILed.

@ericl
Copy link
Contributor Author

ericl commented Nov 27, 2018

Ping @richardliaw

@AmplabJenkins
Copy link

AmplabJenkins commented Nov 27, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9630/
Test FAILed.

richardliaw
richardliaw reviewed Nov 28, 2018
View reviewed changes
richardliaw
richardliaw reviewed Nov 28, 2018
View reviewed changes
python/ray/rllib/evaluation/tf_policy_graph.py
if self._prev_reward_input is not None and prev_reward_batch:
builder.add_feed_dict({self._prev_reward_input: prev_reward_batch})
builder.add_feed_dict({self._is_training: is_training})
builder.add_feed_dict({self._is_training: False})
Copy link
Contributor

@richardliaw richardliaw Nov 28, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where in the code is _is_training True?

Copy link
Contributor Author

@ericl ericl Nov 29, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ray/python/ray/rllib/evaluation/tf_policy_graph.py

Line 227 in dedd3a8

builder.add_feed_dict({self._is_training: True})

richardliaw
richardliaw reviewed Nov 28, 2018
View reviewed changes
Copy link
Contributor

@richardliaw richardliaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few questions

@AmplabJenkins
Copy link

AmplabJenkins commented Nov 29, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9660/
Test FAILed.

@AmplabJenkins
Copy link

AmplabJenkins commented Nov 29, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9661/
Test FAILed.

@ericl ericl merged commit 07d8cbf into ray-project:master Nov 29, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@richardliaw richardliaw richardliaw approved these changes

Assignees

@richardliaw richardliaw

Labels

tests-ok The tagger certifies test failures are unrelated and assumes personal liability.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ericl @AmplabJenkins @richardliaw